Here I show three diagnostics: Convergence and efficiency, Traceplots, and Pairs plots.
From the Stan documentation on convergence and efficiency diagnostics for Markov Chains:
The Rhat function produces R-hat convergence diagnostic, which compares the between- and within-chain estimates for model parameters and other univariate quantities of interest. If chains have not mixed well (ie, the between- and within-chain estimates don’t agree), R-hat is larger than 1. We recommend running at least four chains by default and only using the sample if R-hat is less than 1.05. Stan reports R-hat which is the maximum of rank normalized split-R-hat and rank normalized folded-split-R-hat, which works for thick tailed distributions and is sensitive also to differences in scale.
The ess_bulk function produces an estimated Bulk Effective Sample Size (bulk-ESS) using rank normalized draws. Bulk-ESS is useful measure for sampling efficiency in the bulk of the distribution (related e.g. to efficiency of mean and median estimates), and is well defined even if the chains do not have finite mean or variance.
The ess_tail function produces an estimated Tail Effective Sample Size (tail-ESS) by computing the minimum of effective sample sizes for 5% and 95% quantiles. Tail-ESS is useful measure for sampling efficiency in the tails of the distribution (related e.g. to efficiency of variance and tail quantile estimates).
Both bulk-ESS and tail-ESS should be at least 100 (approximately) per Markov Chain in order to be reliable and indicate that estimates of respective posterior quantiles are reliable.
In the plots below, I will show these three diagnostics for the models run for each of our six populations.
Given that each model is estimating hundreds of parameters, traceplots for all parameters will not be shown here. Instead, traceplots for a few parameters from each model will be shown here.
Pairs plots show univariate histograms and bivariate scatter plots for selected parameters, and are especially useful for identifying collinearity between variables (which manifests as narrow bivariate plots) as well as the presence of multiplicative non-identifiabilities (banana-like shapes).
For these pairs plots, I am showing all of the different parameters that exist for a single movement (e.g., the intercept, origin, and temperature parameters that govern the probability of a single movement, such as ascending a specific dam). Again, there are too many movements across the six models to show all of these plots, but I will show examples of potential issues related to collinearity in our predictors. A description of collinearity of predictors in regressions can be found in the Stan User Guide.
After making the change such that each fish receives only either a DPS-specific or an origin-specific intercept term (which fixed nonsensical collinearity issues between origin and intercept parameters), there are still two common cases of collinearity that are important to be aware of for interpreting parameter estimates: